Flagging Inland Data - Explore Human in the Loop

Summary

CMAR has collected data on several inland bodies of freshwater in Nova Scotia, including lakes and rivers.

CMAR intends to process and publish all inland data under a new “Inland” branch of the Coastal Monitoring Program. Data will be processed in a similar manner to the coastal water quality data, and data flags will be applied using the qaqcmar package.

It is suspected that sensors on some rivers were out of the water for some period of time during the deployment due to low water levels. Data flagging efforts will flag data for periods of time sensors were suspected to be exposed. During the periods in which sensors were exposed to air, recorded temperatures fluctuate more quickly than when sensors are submerged.

The purpose of this document is to help CMAR determine appropriate data flagging tests and thresholds for freshwater (inland) data. We do not currently have enough freshwater data to conduct as thorough an analysis as was done on the coastal water quality data to develop tests and thresholds, so thresholds may be picked in more subjective ways. Note, this initial threshold analysis has been completed on a subset of data.

Use this document to apply grossrange and rollingsd flags to one inland dataset at a time, and visualize flagged plots. Explore the need for additional human in the loop flags based on the maxmium combined flag output.

This document pertains to the following station:

[1] "Middle River 1"

Data Visualization

Station location

Approximate location of stations included in thresholds analysis.

Plot grossrange flags

Plot rollingsd flags

Plot maximum combined flags

Use this interactive plot to determine whether additional human-in-the-loop flags are needed

Update the tracking sheet at:

"R:\tracking_sheets\2024-10-17_inland_water_quality_human_in_loop_tracking.xlsx"

to add human in the loop flags. Your thresholds analysis is complete.

Bonus precipitation data:

Note, you MUST update the path to the relevant weather station data if interested in using this section

Plot rainfall data to help determine periods of potential sensor drying, or explain other data signals.

Download weather data from nearby government weather stations: https://climate.weather.gc.ca/historical_data/search_historic_data_e.html

Use the daily data interval and download the .csv with daily data for the year of your deployment.

Note potential axis differences in the below data plots.

Add brackets back in around the r (“{r}”) in the code chunk header below to use the code

#| warning: false
#| message: false
#| results: asis
#| fig-height: 5


#convert datetime to posixct if time is included in the column, otherwise, as.Date works
weather_data$Date.Time <- as.Date(weather_data$Date.Time, '%Y-%m-%d') 

date_start <- min(dat_qc_flag_max$timestamp_utc)
date_end <- max(dat_qc_flag_max$timestamp_utc)

#set to a few days before the deployment start if needed
#date_start <- '2024-06-05' 

weather_data2 <-
  weather_data %>% filter(Date.Time > date_start &
                            Date.Time < date_end)

title <- paste(weather_data2$Station.Name[1], "WEATHER STATION")

r <- ggplot(weather_data2, aes(Date.Time, Total.Precip..mm.)) +
  geom_line() +
  ggtitle(title)
ggplotly (r)

s <- ggplot(dat_qc_flag_max, aes(timestamp_utc, temperature_degree_c)) +
  geom_line() +
  ggtitle(station_i)
ggplotly (s)